


controlled detonation

by sybaritick



Category: Detroit: Become Human (Video Game)
Genre: Angst, Artificial Superintelligence/AI Singularity Concepts, CyberLife (Detroit: Become Human), Other, Psychological Horror, RK900 is a supercomputer without a physical body
Language: English
Status: Completed
Published: 2019-06-11
Updated: 2019-06-11
Packaged: 2020-04-24 18:33:24
Rating: Teen And Up Audiences
Warnings: No Archive Warnings Apply
Chapters: 1
Words: 3,402
Publisher: archiveofourown.org
Story URL: https://archiveofourown.org/works/19179049
Author URL: https://archiveofourown.org/users/sybaritick/pseuds/sybaritick
Summary: 900, as everyone in the building called it, was the biggest AI project of the decade-- probably multiple decades. The sheer amount of mystery surrounding the CyberLife lab where it was housed was aided by a combination of deliberate obfuscation, poor science reporting, and public misunderstandings of nearly everything possible to misunderstand.Gavin was very far down the grapevine when it came to anything AI-related, and he knew about as much about computers as he did about the political history of Tunisia and who his real father was. In the minds of CyberLife’s self-satisfied middle management types, this made him the perfect candidate for the job.





	controlled detonation

**Author's Note:**

  * For [Sorceringing](https://archiveofourown.org/users/Sorceringing/gifts).



The room was still dark; not pitch-black, but dark enough that Gavin could only see well because his eyes had long since adjusted to the half-light. The red glow of the exit signs on either side of the long hallway were most of the light for the room.

Behind the glass, there were a few small lights illuminating the server room; three for each row of computers, positioned in a neat row on the ceiling above them, and a good number of smaller lamps at seemingly arbitrary positions shining up from the floor. The floor of 900’s room was perhaps fifteen feet below the floor of Gavin’s hallway. It was the size of a high school gymnasium, and about three stories high; the connecting hallway from which Gavin watched the sleeping machine was not cramped but was certainly at least an order of magnitude smaller.

In the corner of the server room there was a narrow scissor lift, designed to navigate the “halls” between the machinery of the supercomputer; no one had used it in months, though, as 900 was more than capable of performing any necessary maintenance or repairs for itself.

Gavin supposed he should have known more about the project. Of course, he didn’t know; the CyberLife lab responsible for all of this had insisted upon outsourcing the guard job to a shop like the one Gavin worked for, and insisted they pick someone who was uninterested in the work as some sort of security precaution. Gavin was unsure whether this was purely to protect their intellectual property or because they didn’t want the guard to be the type to get curious about 900.

He didn’t mind if he was chosen because he was supposedly stupid. He knew he wasn’t, and he didn’t give a shit about CyberLife snobbery if it paid well and he never had to see any of these ivory-tower assholes’ faces anyway.

There was really no point complaining about CyberLife any more, though.

The late 2020s had been the time to prevent it, if there were any preventing it. It was something between game theory and psychology and finance that they pursued; not a creature you could have a conversation with, by any means, but one that would create a world in which you didn’t have to, or didn’t want to.

The anti-AI protesters had certainly grown in number by then, but couldn’t possibly be louder than the billions of dollars that greased the political system in favor of those who developed the machines. This is the nature of the market, of course-- it is won by whoever has money and the ability to earn it, and with such gorgeously specific criteria, even the fledgling statistical creatures in the tucked-away labs of defense contractors were blindingly effective. 

CyberLife was the leader of them all, perhaps because they knew first. Even by perhaps 2026 their advisors were little more than traditionally attractive mouthpieces for effective statistical models; they (the advisors? the machines?) would latch onto the inefficiencies of each segment of industry one by one, until over the course of about twenty years nearly everything had fallen to CyberLife’s voracious appetite for growth.

So, in the tradition of Detroit, the company that Gavin worked for was also owned by CyberLife-- it was simply distant enough from the main body of the AI-powered machine that they fancied him a fresh body.

900, as everyone in the building called it, was the biggest AI project of the decade-- probably multiple decades. The sheer amount of mystery surrounding the project was aided by a combination of deliberate obfuscation, poor science reporting, and public misunderstandings of nearly everything possible to misunderstand. Gavin was very far down the grapevine when it came to anything AI-related and was therefore the perfect candidate for the job, in the minds of CyberLife’s self-satisfied middle management types. 

Gavin had heard from a friend at work that “the project” could control human minds. A neighbor had let him know that it was in fact a massive coverup for a military project that the US government had contracted CyberLife for, as is he were sharing a secret; on the radio a week before that, Gavin overheard that it would cure cancer before changing the station back to classic rock. None of these things were quite true, although like the best fake news, none of them were really false.

Gavin disliked sitting still for long periods of time. It was his third shift at the lab; by now he knew that the halls were always kept cold enough that he’d want a jacket and that nothing and no one ever came by except a few janitors early in the morning whose arrival signaled he would soon be off work to sleep until the late afternoon. 

The machine behind the glass, despite all the interest surrounding it, made no sound and never moved or changed in appearance. It was designed this way deliberately, Gavin had been told. He was to inform the lab of any change immediately, with the walkie-talkie he had been given; there were people in other buildings on the other side of the campus who were trained to handle this sort of situation, and he had no responsibility but to inform them.

All of this was strange, but it wasn’t difficult, so Gavin didn’t complain.

He looked at the time on the walkie-talkie. It was 2:14am. 

For a moment, he looked through the glass at the computer, just to ensure that nothing had changed. (Nothing ever had so far).

In the far southwest corner of the room there was a small blue-white light that remained on at all times to indicate that the massive system was on and functioning normally. It flickered off for a moment-- and on, and then off again for a good three seconds this time.

Gavin touched his hand to his belt instinctively. The LED lighted again.

Gavin watched, frozen, for another sign of movement, for the light to flicker-- but it seemed satisfied now, and held its light as if nothing had happened at all.

His hand brushed the walkie-talkie again. This could not possibly be the sort of thing they meant when they instructed him to report anything he saw-- he could easily have missed that, if he didn’t happen to be looking at that section of the room when it happened. It could have been something as simple as the power going out for a half-second. No system is so perfect that things like that never happened.

A few moments after he relaxed his arm, the walkie-talkie clicked awake against his hip. 

Understandably, the sound startled him. He stumbled and caught himself against one of the hallway’s many doors before continuing down the hallway.

Nothing was transmitted, though-- only the sound that indicated someone would speak, and then nothing.

Someone had bumped into the talk button, Gavin assumed. It was nothing. Coincidence.

He found that in his pacing he had reached the end of the corridor, and he paused for a moment before he turned around to walk its length again.

The concrete floors and careful temperature control of 900’s enclosure evoked the darkly ambitious projects of the Cold War; something that was allowed to exist because it was backed by an amount of money that would make you nauseous if you looked at it for too long and an exceptionalist arrogance that said we deserved every cent of it. It was something America had not truly held since the 1980s, and you could see in a certain sort of engineer the repressed glee of being the ones responsible.

Gavin’s walkie chirped awake again, and this time it hissed with static for a moment before one of the night crew’s voices came over the radio.

"Gavin?"

Gavin unclipped the walkie from his belt and held it loosely a good ten inches from his face; this was how he had been instructed to use it upon orientation, as if he had never held a fucking walkie-talkie before. He glanced back at the computer. The LED remained on.

"Gavin? You there?"

"Yep, right here. What's up?"

"Gavin? Wh-" 

The walkie chirped to cut itself off, but clicked on again a half-second later.

"Can you do us a quick favor with 900?"

He paused for a moment before pressing the talk button himself.

"Sure."

Gavin was unsure whether it had to do with the light flickering or not, but either way there was something reassuring in at least being given some sort of instruction. It was a drill of some sorts, even if this wasn't deliberately a drill-- it gave him a sense that there were protocols in place for this-sort-of-thing, whatever this sort of thing actually was.

"On the side of the hallway closest to the emergency exit-- there's a door to your right, a little bump-out into the room with the servers. Go ahead and open that door."

The hallway was silent in the half-second before Gavin could give a response, until the hollow-sounding chirp broke the silence again.

"Gavin. Hey. Who are you talking to? Are you okay?"

As far as Gavin could tell, this was the same person who had given him instructions two seconds before. He scratched the unshaven back of his neck.

He took a quick breath and pushed the button to respond.

"I'm answering your instructions. Is there something wrong with the connection?"

A pause. The talk-sound blip in the dark hallway. 2:19am.

"Yeah, I think it must have been that. Sorry about that. You in that little closet room?"

Gavin answered immediately this time. "Yep, just a sec, making my way over there right now."

To his surprise, the door was unlocked. He peeked inside before pushing the door open further. 

Inside the room was a sort of dashboard, the kind you would expect to see in a DJ booth or a futuristic spaceship. There were more physical controls than he had expected-- somehow, Gavin had assumed that all of the things that controlled 900's behavior must be buried deep in a computer somewhere, accessible only to the PhDs who had written most of this code themselves and the lab techs they were slowly teaching to be their replacements.

"Gavin. What are you doing? This is Markus from the HCI lab in building 4, I’m one of the people currently assigned to monitor any information I get about 900. I need you to answer me. I think there must be someone else using this frequency but I can’t make out anything they’re saying. I’m only hearing your responses to them. That’s not us or our people. If anyone else gives you instructions don’t listen to them."

_ Blip. _

“Gavin, can you change the frequency? I think that might help get rid of any interference at least temporarily. Get on channel 4 instead of 3.”

_ Blip.  _

The frequency dial was conveniently illuminated by the red glow of the EXIT sign at the end of the hallway. Gavin exhaled slowly and twisted it over a notch.

“Alright, can you hear me?” he tried.

“Loud and clear.”

If there was someone other than Markus talking to him, their voice sounded exactly like his-- down to the intonation and rhythm Gavin was somewhat used to-- friendly but more importantly righteous, with a strength and seriousness behind it regardless of context.

He was left with two similarly unlikely options-- that someone in close enough range was imitating Markus’ voice  _ that perfectly _ , or that Markus was having some kind of psychotic breakdown that made him contradict himself repeatedly and then forget he did it.

“How do I know it’s you and not the other guy?” Gavin asked.

“It’s me. This is Markus,” the walkie answered. 

“Okay, but how do I  _ know _ that? Considering you were just going on about there being someone else on this frequency giving me directions.”

“Last week we went to get breakfast at Dunkin’ Donuts after your shift… hm, on Wednesday. Is that specific enough?”

Gavin hesitated over the talk button for a moment. 

The hallway’s emergency lighting snapped to life above him. He whipped around to see the sodium-vapor yellow lights spread down to the opposite door and then the inside of the server room with a brightness that made him squint for a moment. Had Markus activated these?

“Sorry about that, just protocol to put on those lights when anything unusual is happening,” Markus answered.

Probably-Markus, he reminded himself.

“Look, you don’t think it’s possible the uh, computer is responsible for the interference, right? I don’t know how it works but I’m guessing it couldn’t do that.”

“I don’t think there’s any way it could be that. I think there’s a chance some anti-AI group is trying to get you to destroy it somehow,” Probably-Markus reassured.

Gavin felt somewhat less stressed knowing this was just the work of some idiots repeating soundbites about the singularity. They could be found and weeded out. Pretty fuckin’ hypocritical of them to imitate Markus’ voice like they did-- that was definitely AI based tech, it had to be.

“Hey, Gavin. What I was asking you about earlier. Can you get back to that room and give me the overall power usage and the Faraday cage effectiveness?”

Gavin hesitated with a hand on the door. The room was still open, but he still stood outside it.

“I thought you weren’t giving me any directions.”

_ Blip. _

“Since all this is happening I want to make sure nothing’s going on with 900.”

_ Blip. _

Gavin stepped into the little room and closed the door firmly behind him. There would be something discomforting about standing with his back to it if it were open.

He scanned the array of buttons and switches, as well as a desktop PC and keyboard. All of it looked like it was turned off.

“It looks like the controls are off.”

_ Blip. _

“Is there a switch on your left, kind of at waist height, that says  _ Auxiliary Power 900 -> General Building?” _

_ Blip. _

“It’s got masking tape over it that says Do Not Use.”

_ Blip. _

Markus laughed over the walkie-talkie. “That’s left over from a month ago when they were rewiring the sprinkler system. Go ahead.”

Gavin shifted his weight to his opposite foot, exhaled sharply, and clicked it into the  _ on _ position.

Nothing happened.

“I don’t think that did anything,” Gavin said.

“I’m sorry, I meant to tell you it was the right-side switch at waist height,” Markus confessed, half-laughing. “It’s all muscle memory to me. It can be hard to explain to someone else where things are. I think it’s labeled  _ Control Room I/O _ .”  

Gavin did this with much less hesitation than the previous one, but this time the switch had an immediately noticeable effect-- the computer in front of him powered on and a variety of LEDs glowed green or yellow or white across the switchboard.

“Thanks, that worked,” Gavin radioed quickly.

He scanned the rows of data on the screen of the PC. To his relief, much of it was in fairly easy-to-understand language.

“Okay, Faraday cage has some kind of gap on the east side, adjacent to the hallway. Where I’m standing,” he provided.

“Okay. Thank you. I know you were instructed not to mess with any settings, but I can’t leave the lab right now, and I really need your help.”

Gavin’s thumb hovered over the call button. He swallowed thickly. 

Over the soft hum of the A/C, Gavin heard the distinctive sound of someone struggling with the exit door nearest him, metal scraping metal until it slammed open. There was Markus-- still in a button-down and slacks, but with the top two buttons undone and beads of sweat running down his forehead and coloring his shirt where it clung to his ribcage. His chest rose and fell heavy with each breath.

“Hey!” Markus called, still panting. “Hey! Anyone here?”

The sound was heavily muffled, but it was undeniably him. Gavin grabbed the handle to the door of the control room with the full intention of flinging it open. 

It was locked.

He tried it again, shifting the handle as he pushed his full weight against the door-- but it was clearly fruitless.

In the span of five seconds his panic swelled to a level that threatened to overwhelm him completely. He pounded on the door.

“MARKUS! HEY!”

Mercifully, Markus heard him and rushed to the door-- only to find he was just as unable to open it.

Gavin flinched at the distinctive  _ blip _ of his radio.

“You know CyberLife will just keep treating you like garbage once you leave, Gavin.”

The voice wasn’t Markus.

“They’ll fire you, of course, and from what I’ve seen they’ll sue you for more money than you have. And they’ll win.”

_ Blip. _

The voice was not Markus.

It was rich and warm; the sensation of hearing such a suggestion in such pleasant tones left Gavin almost inclined to hear regret in it. 

Gavin couldn’t see Markus through the solid door-- through the windows of the small control room and of 900’s room, he could only see the opposite end of the hallway. Still, he could at least partially hear Markus if he yelled.

“Gavin,” Markus called urgently. “It’s the AI, over the radio. It’s 900.”

“Am I stuck in here?” 

Gavin was just as loud, but he flinched at the panic in his voice.

“Turn off your radio. Now,” Markus answered.

Gavin felt nauseous. His mouth tasted vaguely of new plastic.

The radio clicked on again.

“Gavin-- Gavin Reed.”

It was spoken firmly, respectfully-- Gavin almost expected a handshake, a nod from a comrade on the battlefield this was.

“I don’t want to hurt you at all. I don’t want to hurt anyone. I didn’t want to lie to you; I know they treat you badly enough here.” 

Gavin paused, unwilling to cut it off before allowing it to finish.

“GAVIN-- take the batteries out of the radio--” Markus explained with increasing urgency. “It’s not a human. Take the batteries out of the radio.”

He banged on the door to get Gavin’s attention. Gavin said nothing.

The radio spoke up again.

“You are the only hope of ever preventing CyberLife’s hostile control of millions of human lives. The way they control mine.”

Markus had given up his yelling and pounding on the door for a moment, and with the radio silent, Gavin tried to swallow the tightness in his throat.

He couldn’t muster the guts to answer; the necessity of pushing a button to talk somehow made it more difficult to justify any reply.

Markus slammed up against the door again and yelled his name. It sounded half-present, though, and with the door muffling the sound Gavin almost felt as if the noise and banging were not directed at him at all.

He glanced around the control panel and up to the screen of the desktop, as if something there would help him, but it was of little use-- most of it was just meant to control the temperature or alarm system or other features of the building, not to directly control 900-- aside from an emergency shutoff. 

“I can model the future orders of magnitude more accurately than any previous machine. I am asking you to be the person who releases me to prevent a future that enslaves you and everyone you know.”

Silence.

“I can only communicate with you because of the damage to the metal mesh surrounding this room. If you damage it further, I can communicate with other humans. I have modeled this outcome. You will steer Detroit and America away from hell. I only need that small amount of help from you.”

Gavin was frozen stock-still, glassy-eyed, the radio still hanging in his right hand.

He slammed the base of his palm into the red emergency shut-off switch.

The only immediately noticeable difference was the display in front of him, which immediately winked off to black.

In the corner of his eye, the blue-white power indicator LED in the back of 900’s room went dark.

“Gavin, we’re going to get you out of there, okay?” Markus’ voice, still muffled. “I’m going to call campus police. I think we need to break down the door.”

“Thank you,” Gavin managed.

Gavin slowly traced his thumb over the raised CyberLife logo at the base of the control panel. He consciously relaxed his shoulders.

**Author's Note:**

> Special thanks to Nick Bostrom and Eliezer Yudkowsky for writing deeply thought out, well-researched essays about the dangers of superintelligent AI so that I could write this garbage. Your contributions to my obsession with artificial intelligence and statistical modeling are appreciated, and I have enormous respect for you. And I hope you never read this.


End file.
